IN THE CLAIMS : 

Please cancel Claims 1-3 and 5-8 without prejudice to or disclaimer of the subject 
matter presented therein. 

Please amend Claims 15-21 and add new Claims 22-26 as follows. 

1.-3. (Cancelled) 

4. (Cancelled) 

5. - 8. (Cancelled) 
9.-14. (Cancelled) 

15. (Currently Amended) A mixed reality space image generation method for 
generating a mixed reality space image which makes a player experience mixed reality by 
mixing a real space in which markers serving as position indices are laid out, and a virtual 
space, comprising: 

a marker detection step of detecting the markers from image data obtained by sensing 
the real space from a substantially viewpoint position of the player; 

a correcting step of correcting location/posture orientation information of the player 
based on the markers detected by said marker detection step; and 

a mixed reality space image generation step of generating a mixed reality space image 
to be observed by the player, so the player observes virtual object images that do not 
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include any images of the markers in marker surrounding regions (marker regions) 
including the markers from in the image data, 

wherein said the mixed reality space image generation step includes a step of 
substituting or overlaying images of the marker regions by predetermined virtual object 
images. 

16. (Currently Amended) A mixed reality space image generation method for 
generating a mixed reality space image which makes a player experience mixed reality by 
mixing a real space in which markers serving as position indices are laid out, and a virtual 
space, comprising: 

a marker detection step of detecting the markers from image data obtained by sensing 
the real space from a substantially viewpoint position of the player; 

a correcting step of correcting location/ p o sture orientation information of the player 
based on the markers detected by said the marker detection step; and 

a mixed reality space image generation step of generating a mixed reality space image 
to be observed by the player, so the player observes virtual object images that do not 
include any images of the markers in marker surrounding regions (marker regions) 
including the markers from the image data, 

wherein said the mixed reality space image generation step includes a step of 
substituting or overlaying an image of a region including all the markers in the image data 
with by a predetermined virtual object images image . 
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17. (Currently Amended) The method according to claim 15, wherein the 
predetermined virtual object objects are images are imag e s of plane patches with the same 
or similar texture, size, location and posture orientation of the respec t ive marker regions 
before the markers are laid out. 

18. (Currently Amended) A computer readable storage medium that stores a program 
which can be executed by a computer, said program making the computer perform the 
steps of which executes the program function as : 

[[a]] marker detection step o f means for detecting the markers , serving as p o si t i o n 
indices, from image data obtained by sensing [[a]] the real space from a substantially 
viewpoint position of [[a]] the player; 

[[a]] correcting step o f means for correcting location /posture orientation information 
of the player based on the markers detected by said the marker detection step means ; and 

[[a]] mixed reality space image generation step of means for generating a mixed reality 
space image to be observed by the player, so the player observes virtual object images that 
do not include any images of the markers in marker surrounding regions (marker regions) 
including the markers from in the image data, 

wherein said the mixed reality space image generation step means includes means for 
substituting or overlaying images of the marker regions by predetermined virtual object 
images. 
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19. (Currently Amended) A computer readable storage medium that stores a program 
which can be executed by a computer, said program making the computer which executes 
the program function as perform the steps of : 

a mixed reality apparatus for making a player experience mixed reality by making the 
player observe a mixed reality space image obtained by mixing a real space in which 
markers serving as position indices are laid out, and a virtual space, comprising: 

[[a]] marker detection step o f means for detecting the markers se r ving as p o siti o n 
indices from image data obtained by sensing [[a]] the real space from a substantially 
viewpoint position of [[a]] the player; 

[[a]] correcting step f or means correcting location/ posture orientation information of 
the player based on the markers detected by sard the marker detection step means : and 

[[a]] mixed reality space image generation step-of means for generating a mixed reality 
space image to be observed by the player, so the player observes virtual object images that 
do not include any images of the markers in marke r surrounding regions (marker regions) 
including the markers from in the image data, 

wherein said the mixed reality space image generation step means includes means for 
substituting or overlaying an image of a region including all the markers in the image data 
by a predetermined virtual object images image . 

20. (Currently Amended) A mixed reality apparatus for providing a mixed reality 
space shared by a plurality of players, comprising: 

an input unit adapted to input a real space image representing the real space; 
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a marker detecting unit adapted to extract pixels having a color that is predefined as a 
marker from the real space image, perform a labeling process to detect marker regions, and 
detect positions of markers [[in]] from the marker regions; 

a head location /p o sture orientation estimation unit to estimate a location/ posture 
orientation of a viewpoint of a given player by using the detected marker positions; 

a mixed reality space image generation unit adapted to generate a virtual space image 
to be observed by the given player based on the location/ posture orientation of the 
viewpoint of the given player and model information; and 

an image combining unit adapted to generate a mixed reality space image by 
combining the real space image with the virtual space image, 

wherein the markers are laid out in the real space so that the markers to be used by 
only the given player are laid out at positions hidden by real objects when the markers are 
observed from the other player. 

21 . (Currently Amended) A method for providing a mixed reality space shared by a 
plurality of players, comprising: 

an input step of inputting a real space image representing the real space; 

a marker detecting step of extracting pixels having a color that is predefined as a 
marker from the real space image, performing a labeling process to detect marker regions, 
and detecting positions of markers [[in]] from the marker regions; 

a head location/posture orientation estimation step of estimating a location /postu r e 
orientation of a viewpoint of a player by using the detected marker positions; 
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a mixed reality space image generation step of generating a virtual space image to be 
observed by the player based on the location/ posture orientation of the viewpoint of the 
player and model information; and 

an image combining step of generating a mixed reality space image by combining the 
real space image with the virtual space image, 

wherein the markers are laid out in the real space so that the markers to be used by 
only a given player are laid out at positions hidden by real objects when the markers are 
observed from the other player. 

22. (New) A mixed reality apparatus having an image generating unit adapted to 
generate a virtual space image corresponding to a viewpoint position and orientation of a 
player, an input unit adapted to input a real space image obtained by sensing from a 
substantially viewpoint position of the player and an image combining unit adapted to 
generate a mixed reality space image by combining the real space image with the virtual 
space image, comprising: 

a marker detection unit adapted to detect marker regions for the real space image input 
from said input unit and obtained by sensing the real space in which markers serving as 
position indices are laid out; 

a correction unit adapted to correct location/orientation information of the player based 
on the marker regions detected by said marker detection unit; 

a location/orientation estimation unit adapted to estimate a location/orientation of a 
viewpoint of a given player using the detected marker regions; 
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a storing unit adapted to store a real space image obtained by sensing the real space 
before the markers are laid out as a texture image; and 

a replacing unit adapted to replace said marker regions in the real space image with 
corresponding regions in said texture image or superimpose said corresponding regions in 
said texture image on said marker regions in the real space image. 

23. (New) A mixed reality image generating method including an image generating 
step of generating a virtual image corresponding to a viewpoint position and orientation of 
a player, an input step of inputting a real space image obtained by sensing from a 
substantially viewpoint position of the player and an image combining step of generating a 
mixed reality space image by combining the real space image with the virtual space image, 
comprising: 

a marker detection step of detecting marker regions for the real space image obtained 
by sensing the real space in which markers serving as position indices are laid out; 

a correction step of correcting location/orientation information of the player based on 
the marker regions detected by said marker detection step; 

a storing step of storing a real space image obtained by sensing the real space before 
the markers are laid out as a texture image; and 

a replacing step of replacing said marker regions in the real space image with 
corresponding regions in said texture or superimposing said corresponding regions in said 
texture image on said marker regions in the real space image. 
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24. (New) The mixed reality generating method according to claim 23, wherein the 
same texture image is used for replace or superimpose the marker regions corresponding to 
markers laid out on real objects made of an identical material. 

25. (New) The mixed reality image generating method according to claim 23 further 
comprising: 

a location/orientation estimation step of estimating a location/orientation of a 
viewpoint of the player using the detected marker regions, the location information of the 
detected markers and output signal from a position/orientation sensor mounted on the 
player's head. 

26. (New) A computer-readable storage medium that stores program code for 
controlling an apparatus to perform a mixed reality generating method, wherein the 
generating method includes an image generating step of generating a virtual image 
corresponding to a viewpoint position and orientation of a player, an input step of inputting 
a real space image obtained by sensing from substantially a viewpoint position of the 
player and an image combining step of generating a mixed reality space image by 
combining the real space image with the virtual space image, said storage medium 
comprising program code for controlling the apparatus to perform: 

a marker detection step of detecting marker regions for the real space image obtained 
by sensing the real space in which markers serving as position indices are laid out; 

a correction step of correcting location/orientation information of the player based on 
the marker regions detected by said marker detection step; 
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a storing step of storing a real space image obtained by sensing the real space before 
the markers are laid out as a texture image; and 

a replacing step of replacing said marker regions in the real space image with 
corresponding regions in said texture or superimposing said corresponding regions in said 
texture image on said marker regions in the real space image. 
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